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Abstract 

In this paper, we study the commuting variety of symmetric pairs 
associated to parabohc subalgebras with abelian unipotent radical 
in a simple complex Lie algebra. By using the "cascade" construc- 
tion of Kostant, we construct a Cartan subspace which in turn pro- 
vides, in certain cases, useful information on the centralizers of non 
p-regular semisimple elements. In the case of the rank 2 symmetric 
pair (sOp+2,sOp x S02), p > 2, this allows us to apply induction, in 
view of previous results of the authors and reduce the problem 
of the irreducibility of the commuting variety to the consideration of 
evenness of p-distinguished elements. Finally, via the correspondence 
of Kostant-Sekiguchi, we check that in this case, p-distinguished el- 
ements are indeed even, and consequently, the commuting variety is 
irreducible. 

2000 Mathematics Subject Classification 17B20, 14L30 

1 Introduction and notations 

Let g be a complex simple Lie algebra and 6 an involutive automorphism of 
g. Let g = 6 © p be the decomposition of g into eigenspaces with respect to 
e, where t = {X e 5 \ e{X) = X},p = {X Eq] e{X) = -X}. In this case, 
we say that (g, 6) is a symmetric pair. 
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Let G be the adjoint group of g and K the connected algebraic subgroup 
of G whose Lie algebra is t. 

Let a be a maximal abelian subspace of p consisting of semisimple ele- 
ments. Any such subspace is called a Cartan subspace of p. All the Cartan 
subspaces are iT-conjugate. Its dimension is called the ran/c of the symmetric 
pair {g,t). 

We define the commuting variety of (g, t) as the following set : 
C(p) = {{x,y) epxp\[x,y] = 0}. 

We may also consider the commuting variety G{g) of g, defined in the 
same way. Richardson proved in [lOj that, if f) is a Cartan subalgebra of 
g, then C(g) = G.(f) x [)). In particular, the commuting variety C(g) is an 
irreducible algebraic variety. 

On the other hand, the commuting variety of any semisimple symmetric 
pair is not irreducible in general. Panyushev showed in |7j that in the case of 
the symmetric pair (sl„, gl„_]^), n> 2, associated to the involutive automor- 
phism, defined via conjugation by the diagonal matrix diag(— 1, ...,—1,1). 
The corresponding commuting variety has three irreducible components of 
dimension, respectively, 2n — 1, 2n — 2, 2n — 2. 

Nevertheless, in some cases, the irreducibility problem has been solved. 

- As an obvious consequence of the classical case proved by Richard- 
son, the symmetric pair (g x g, A(g)), associated to the automorphism 
{X,Y) f-^ {Y,X), has an irreducible commuting variety. 

- If the rank of the symmetric pair (g, fi) is equal to the semisimple rank 
of g (called the maximal rank case), then Panyushev proved in [7j that 
the corresponding commuting variety is irreducible. 

- The rank 1 case has been considered independently by the authors ^T] 
and Panyushev [H]. In this case, it has been proved that (sOm+i,sOm) 
is the only symmetric pair whose commuting variety is irreducible. 

- In |H] , Panyushev proves the irreducibility of the commuting variety for 
the symmetric pairs (sl2n,sp2„) and (£'6,-^4)- 

For a symmetric pair of rank strictly larger than one, we observe that 
due to the rank 1 case, the inductive arguments used by Richardson in the 
classical case jTUj do not apply. However, if a is a Cartan subspace, then it is 
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well-known that Cq = K.{a x a) is the unique irreducible component of C(p) 
of maximal dimension, which is equal to dim p + dim o. The main problem 
is therefore to determine if there exist components other than the maximal 
one. 

In |8j, it has been conjectured that C(p) is irreducible if the rank of the 
symmetric pair is greater than or equal to 2. 

In this paper, we consider symmetric pairs (g, t) associated to parabolic 
subalgebras with abelian unipotent radical. In this case, by using the "cas- 
cade" construction of Kostant, we obtain a particular Cartan subspace of p 
which, in certain cases, turns out to be a useful tool for the description of 
symmetric subpairs of (g, 6), associated to centralizers of semisimple elements 
of p. 

The information on the centralizers of semisimple elements of p allows us 
to apply the induction step of Richardson in the case of the rank 2 symmetric 
pair (sOp+2,sOp x S02), p > 2. As a result, we obtain that all the irreducible 
components of C(p) other than Cq are necessarily related to /C-orbits of 
p-distinguished elements in p. 

By using the Kostant-Sekiguchi correspondence, we are able to estab- 
lish that in the case of symmetric pair (sOp+2,sOp x S02), p > 2, every p- 
distinguished elements is even. We prove finally that this condition is suf- 
ficient for the irreducibility of the commuting variety of (sOp+25SOp x SO2), 
p>2. 

We shall conserve the notations above in the sequel. The reader may 
refer to [12J for basic definitions and properties of symmetric pairs. The 
paper is organized as follows. In Sections |21 and 01 we study symmetric pairs 
associated to parabolic subalgebras with abelian unipotent radical and the 
description of subpairs associated to centralizers of semisimple elements. We 
recall some well-known results relating sheets and the commuting variety in 
SectionlU In particular, we prove that an even nilpotent element of p belongs 
to a sheet containing non-zero semisimple elements. Section |S] is dedicated 
to the rank 2 symmetric pair (sOp+2,sOp x S02), p > 2. Finally, Section IHl 
describes certain computations used in Section |21 
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2 Symmetric pair associated to a parabolic 
subalgebra with abelian unipotent radical 

Let us fix a Cartan subalgebra f) of g and a Borel subalgebra b containing f). 
Denote hy R D D H the corresponding set of roots, positive roots and 
simple roots. Let us also fix root vectors Xq,, a E R, and for a E R, we set 

fla = CXa- 

For any subset S* C LI, we denote hj Rs = Rr\ ZS. Then Rs is a root 
system in the vector subspace that it spans in f)*. Moreover is a base of 
simple roots for Rs- 

Set = i?s n i?^, and 

qs = [) © 3a 

aeRs'JR+ 

the standard parabolic subalgebra associated to S. The unipotent radical Us 
of qs is therefore 0„g^i Qa where R^ = R'^ \ Rs- 

Let C n be such that the unipotent radical of qs is abelian. Then 
it is easy to check that qs is maximal. Moreover, if we set % = ©Q,g/ji Q-a, 
the decomposition 

= U5 © ({) © 0a) © % 

induces a Z-grading on q. This in turn defines a symmetric pair with 

^5 = [) © 0a , P5 = U5 © %. 

aeRs 

These symmetric pairs correspond to inner automorphisms of with non 
semisimple fixed point sets. A list of all such parabolic subalgebras and the 
corresponding symmetric pairs is given in Table 1 (see also Table 7 of the 
Reference chapter of P). 

We shall construct a nice Cartan subspace for these symmetric pairs by 
using the following properties of the root system, sometimes referred to as 
the "cascade" construction of Kostant. We shall recall this construction and 
certain useful properties. For more details, the reader may refer to j^, [S] or 

m 

Let Ten. If Rt is irreducible, then T is connected, and we shall denote 
by Et the highest root of Rt- 

We define a set /C(T) by induction on the cardinality of T as follows : 
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• /C(0) = 0. 

• If Ti, . . . , are the connected components of T, then : 

/C(T) =/C(ri)U---U/C(T^). 

• If T is connected, then 

/C(r) = {T} U /C({a e T ; a and £t are orthogonal}). 

Note that any K G /C(T) is connected, and the roots Ek, K E IC(T), are 
pairwise strongly orthogonal (two distinct positive roots a, {3 are strongly 
orthogonal if a ± /5 are not roots). 

For K e JC{T), denote by the set of positive roots a e i?^ which are 
not orthogonal to Ek- 

We have the following properties of the sets . 

• — \Jk£k.{t) (disjoint union). 

• Let K G IC(T). If a G \ {sk}, then there exists a unique j3 G 
such that a + j3 = ek- 

• Let K, K' G /C(T), a G and /3 G F^' be such that a + /3 G i?, then 
either K C i^' and a + /3 G F^' or iT' C K and a + /3 G F^. 

Let 5* C n be such that Ug is abelian. We set 

8 = {Ke /C(n) ; EK e R^}, 

and for K E S, Xx = X^-^ + X_^-^. Note that the elements X^ arc semisim- 
ple. Let (g, €5) be the corresponding symmetric pair as defined above. 

Proposition 2.1 1. We have R\ (Z[jj^^^V^ . 

2. LetKeS and aeR^n F^. Then Ek - a ^ R\. 

3. The subspace a spanned by the Xk, K E £, is a Carian subspace ofps, 
and the rank of (g, €5) is '^S. 
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Proof. The first two parts are easy consequences of tlie properties of T 
stated above and of the fact that U5 is an abehan ideal of qs- 

Since the Ek, K ^ £, are pairwise strongly orthogonal, the subspace a of 
ps consists of pairwise commuting semisimple elements, and its dimension 
is ^8. Using Table 1, we check that a has the right dimension for a Cartan 
subspace. ■ 

We list below all the standard parabolic subalgebras qs with abelian 
unipotent radical, the unique simple root in Rg, the corresponding symmetric 
pair, and the elements of the set S. We follow the numbering of simple roots 
in p. 





symmetric pair 


elements of S 




(sU+i,sl„+i_i X sli X C) 


{aj, . . . , ttn+l— j}) 


n > 1 




1 < j < min(i, n + 1 — i) 


(5„,Q!i) 


(s02„+l,S02n-l X SO2) 


{ai},U 


n>2 






{Cn, an) 


(sP2n,gln) 


m) 


n>2 








(s02n,S02„-2 X SO2) 


{ai},n 


n > 4 






(Dn^ai), n>4: 


(S02„,gl„) 


{am, • • • , an},rn < n — 2 odd 


i = n — l,n 




and {ctj} if n G 2Z 


{EG,ai), (EQ^ae) 


iEe,D,xC) 


n\{a2}, n 


(Et, aj) 


iEj,Ee X C) 


{ay}, n\{ai}, n 



Table 1 



3 Symmetric pair associated to the central- 
izer of a semisimple element 

In this section, (g, ts) is a symmetric pair associated to a standard parabolic 
subalgebra qs with abelian unipotent radical. Let a be the Cartan subspace 
in p5 as defined in part 3 of Proposition 12.11 
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Let X e a. Then is a Levi factor of a parabolic subalgebra of Q. 
Denote by I = [q^ ,Q^] the semisimple part of , and set [+ = I O , 
L = in p5 and r+ = [L, [_]. Then the decompositions 

0-^ = t-^ © p-^ , [ = 1+ © [_ and r = r+ © l_ 

define symmetric subpairs of (0,65); and the ranks of the pairs ([, [+) and 
(r, r+) are strictly inferior to that of (g, ts)- 

We shall determine, in certain cases, the symmetric pair (r, r+) for any 
non-zero non pg- regular element X & a, i.e. pg contains a non-zero nilpotent 
element. 

To simplify notations, Bq will correspond to the Lie algebra {0}, Bi = Ai, 
1)2 = Ai X Ai and D3 = As. 

We shall need the classification of simple symmetric pairs of rank less 
than or equal to 2. 

List 3.1 Rank 1 ; 

(sl„+i , sl„ X C) , (so„+i , so„) , 

(sP2n>SP2n-l X SP2), (-^4,54). 

List 3.2 Rank 2 ; 

(sln+2,sl„ X SI2 X C), (sl3,S03), (sl6,SP6), (sO„+2, SO^ X SO2) , 

(sP2n+4>SP2n X SP4) , (sP4, X C) , (sOio, slg X C) , (-^6,^4) , 

{Ee,D,xC), (6*2,^1 xAi). 

Lemma 3.3 Suppose that we are in the case {B^, ai) of Table 1. Then there 
exists m e N such that (t, r+) — (so^+i,sOto). 

Proof. From the definition of a, we verify easily that a commutes with the 
Lie subalgebra s generated by the root vectors X±a-, a G 11 \ {cti, a2}. So 
contains 5. Note that 5 is simple of type -B„_2. 

If i is a Cartan subalgebra of the 5, then o ® t is a Cartan subalgebra of 
g^ . It follows that the root system of the semisimple part [ of g^ contains 
as a subsystem the root system of s. In particular, the semisimple rank of [ 
is equal to n — 2 or n — 1. 

Since X is not p-regular, p^ contains a nilpotent element, and so { contains 
strictly s. It follows that the root system R{\) of [ is of one of the following 
types : A^-i, B^-i or Ai x S„_2. 
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Suppose that R{1) is not of type Then contains a simple Lie 

subalgebra of type i?n-2- Consequently, using the classification of rank 1 
symmetric pairs (see List 13. we deduce that r has the required form. 

Suppose now that R{1) is of type An-i. Then from the fact that R{1) 
contains a subsystem of type Bn-2, we deduce that t?, = 3. Explicit com- 
putations as described in the Section El show that this case does not occur. 
■ 

Lemma 3.4 Suppose that we are in the case {Dn, ai) of Table 1. Then there 
exists m G N such that (t, t+) = (sOm+i,sOm)- 

Proof. The proof is more or less the same as in the case ai). However, 
note that s is simple except when n = 4, in which case it is semisimple. The 
root system is of one of the following types : A„_i, Dn-i or Ai x Dn-2- 
Here, if R{1) is of type An-i, then n = 4 or r?, = 5. In the former, we use the 
classification of rank 1 symmetric pairs to obtain a contradication, while in 
the latter, a direct computation is required (see Section IH)). ■ 

Remark 3.5 Note that we may extend Lemma (3.41 to the symmetric pairs 
(so4,so2 X S02) and (so6,so4 x S02). 

In the first case, we have (so4,so2 x S02) = (so3,so2) x (so3,so2), while in 
the second case, we have (so6,so4 x S02) = (sl4,sl2 x SI2 x C). In both cases, 
we may obtain the same conclusion by direct computations (see Section IH)). 

Summarizing, since Cartan subspaces are K-conjugate, we have therefore 
obtained the following result : 

Proposition 3.6 Let (0,6) be the symmetric pair (sOp+2,sOp x S02), p > 2. 
For any non-zero non p-regular semisimple element X in p, the subpair (r, r+) 
is of the form (so^+i, so^) for some m G N. 

Lemma 3.7 Suppose that we are in the case (£"7,0:7) of Table 1. Then 
the symmetric pair (r, r+) is the product of symmetric pairs of the form 
(sOm+i,sOm) or (sOm+2,sOm X SO2) for somc integer m G N. 
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Proof. Proceeding as in the proof of Lemma 13.31 we check easily that 
tg contains the Lie subalgebra s generated by the root vectors X±a, o ^ 
n \ {tti, ctg, Note that s is simple of type D4. 

Hence R{1) contains a subsystem of type D4. We deduce therefore that 
R{1) is of one of the following types : D5, D4 x C, Dq or D5 x Ai. We may 
conclude by using the classification of symmetric pairs of rank less than or 
equal to 2 (see Lists I3.ll and I3.2|) . ■ 

Remark 3.8 Note that in all the other rank 2 cases, there exists X G a non 
p-regular such that p"^ contains two non proportional commuting nilpotent 
elements. Hence by the result of jlll Proposition 3], the conclusion of the 
previous lemmas fails. See Section IHl for more details. 

4 Sheets and commuting varieties 

Let (g, 6) be a symmetric pair. Recall that the connected algebraic group K 
acts on p. For n G N, we set : 

= {X ep; dimK.X = n}. 

The set p'-"-' is locally closed, and an irreducible component of p*^"-* shall be 
called a K -sheet of p. Clearly, X-sheets are X-invariant, and by jT^j, each 
X-sheet contains a nilpotent element. 

Let TTi : C(p) — > p be the projection {X, Y) X. Recall the following 
result concerning the commuting variety of p. 

Theorem 4.1 There exist K-sheets iSi, . . . ,5^ 0/ p such that Hi^^Si), i = 
1, . . . ,r, are the irreducible components of C{p). 

The proof of Theorem 14. II is a simple consequence of the following result. 
For the sake of completeness, we have included a proof. 

Lemma 4.2 Let V be a vector space, E G V x V a locally closed subvariety 
and for i = 1,2, Hi : E V be the projection (xi, X2) ^ Xi. Suppose that : 

1. Tii{E) is locally closed. 

2. There exists r G N such that for all x G tii^E), 'K2{'k^'^{x)) is a vector 
subspace of dimension r. 

If 7Ti{E) is irreducible, then so is E. 
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Proof. Let G be the Grassmann variety of r- dimensional subspaces of V, 
X G 7ri(i?) and W = 7r2(7rf ""^(x)) G G. Fix a complementary subspace U of 
in \^ and set : 

F = {TgG; TnU = {0}} = {T G G ; T + U = V}. 

Clearly, F is an open subset of G containing W. For r G Hom(VF, U) the set 
of linear maps from W to U, we define: 

T(t) = {w + t{w) ; w G PF}. 

Then we check easily that T(r) G F, and we have a map Hom(VF, f/) — »• 
F , r 1-^ ^(t)- We claim that this map is an isomorphism. 

Since Wi + ti{wi) = W2 + T2{w2) is equivalent to W1 — W2 = T2(w^2) — ti(w^i), 
we deduce that the above map is injective. 

Now if T G F, then for w G W, we define t{w) to be the unique element 
in U such that w + t{w) G T. We then verify easily that T(t) = T. So we 
have proved our claim. 

The map 

$:7ri(^)-.G, y^n2{T^^\y)) 

is a morphism of algebraic varieties. So F = $~^(F) is an open subset of 
Tii{E) containing x. The above claim says that we have a well-defined map: 

^! : F yiW ^ E , {y , w) (y , w + t{w)) 

where T(t) = It is then a straightforward verification that \1' is an 

isomorphism of the algebraic varieties F x W and '7rjf^(F). 

It follows that the map ni : E ^ T^iiE) is an open map whose fibers 
are irreducible. Hence by a classical result on topology [HI T.5], if 7ri{E) is 
irreducible, then E is irreducible. ■ 

Since the set of p-generic elements and the set preg of p-regular elements 
are open subsets of p, we have the following corollary : 

Corollary 4.3 Let a be a Cartan subspace in p. The set 

Co = K.{ax a) = vrf ^(prcg) = 7r^^(prcg) 
is the unique irreducible component ofC{p) of maximal dimension. 



10 



We shall finish this section with a result on even nilpotent elements. Let 
X G p be a nilpotent element, and {H, y) G 6 x p be such that (X, H, Y) is 
a normal sl2-triple (called a normal S-triple in [^). Recall that X is even if 
the eigenvalues of a.dgH are even. In fact, this is equivalent to the condition 
that the eigenvalues of adpH are even. 

Proposition 4.4 Let X & p be an even nilpotent element, then X belongs 
to a K-sheet containing semisimple elements. 

Proof. Let (X, H, Y) be a normal slg-triple and s = CX + Ci^ + CY . Then 
g decomposes into a direct sum of s-modules, say V^, i = 1, . . . , r. Since X 
is even, dim Vi is odd for i = 1, . . . , r. 

For A G C, we set Xa = X + XY G p. If A 7^ 0, then Xa is semisimple 
because Xa is G-conjugate to a multiple of H. We claim that dimp"^^ = 
dimp^ for all A G C. 

First of all, observe that p^^ = 0-=i(Vi n p)^^ because Vi = {Vi fl ® 
{Vi n p). Moreover dim(Vi n p)^^ < 1. 

Now if {Vi n p)^^ 7^ {0}, then a simple weight argument shows that 

{Vnp)^y^{0}. 

Conversely, suppose that {Vi n p)^ ^ {0}. Let dimV = 2n + 1 and 
V-n, ■ ■ ■ ,Vn be a basis weight vectors of Vi such that Hvk = 2kvk, k = 
-n,...,n. Then {V H p)^ = Cvn- 

So Vk G t (resp. Vk G p) when n — k is odd (resp. even). In particular, 
v^n G p. It follows that for k such that n — k odd, XYvk+i = —ai^Xvk-i 
for some a^. G C. We may therefore renormalize the v^s so that v = v^n + 
v_n+2 H \- Vn-2 + Vn Verifies Xxv = 0. 

We have therefore proved that dimp"^^ = dimp"^ for all A. 

Now, consider the morphism $ : K x C ^ p, {k, X) k.X\. The image 
of $ is irreducible and contains semisimple elements, so it contains strictly 
K.X. Consequently, K.X is contained strictly in a iC-sheet with semisimple 
elements. ■ 

5 The rank 2 case 

In this section, {q, i) is a symmetric pair of rank 2. 
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Lemma 5.1 Let X ^ p be an element which is non-semisimple and non- 
nilpotent, and X^, Xn G p \ {0} be its semisimple and nilpotent components. 
If{X,Y) G C{p), then the semisimple component ofY belongs to CXg. 

Proof. Let Y = Yg + Ynhe the corresponding decomposition into semisimple 
and nilpotent components. Since = fl g"^", it follows easily that 
= p^" n p^". Hence Xg, X„, Yg, Yn commute pairwise. If Xg and Yg are 
not proportional, then they span a Cartan subspace because the rank is two. 
But this would imply that X commutes with a Cartan subspace, thus X 
belongs to a Cartan subspace, which is absurd because X is not semisimple. 
■ 

In the rest of this section, we consider the rank 2 symmetric pair (g, 6) = 
(sOp+2,sOp X S02), p > 2. In particular, if jo 7^ 2, then the symmetric pair 
comes from a parabolic subalgebra with abelian unipotent radical. 

Let a be a Cartan subspace in p. Recall that Co = K.{a^ a). 

Theorem 5.2 The commuting variety C(p) is irreducible. 

Proof. We proceed as in the proof of Richardson in the classical case (see 
PD] ) by using inductive arguments. Let {X,Y) G C(p). 

1. If X is semisimple, then X commutes with a p-regular semisimple 
element Z. The line Cz = {{^1 tl" + (1 — t G C} is contained in C(p). 
Since {tY + (1 — t)Z,t G C} meets the set of p-regular semisimple elements 
which is open in p, we conclude that Cz-, and hence (X, F), are contained in 
Co (Corollary 1121). 

2. We may assume that neither X nor Y is semisimple. 

Suppose that X is not nilpotent. Writing X = Xs + X„, y = + 1^, we 
deduce from the Lemma IKH that Yg G CXs,X„,y„ G p^^ and [X„,y„] = 0. 

By Proposition 13.61 we deduce that X„ , Yn are commuting elements con- 
tained in a symmetric pair of the form (sOm+i, som) for some m G N. Conse- 
quently, Yn G CX„ by im Proposition 3]. 

Thus, there exists A, G C such that Y = XXg + fiXn. In particular, X 
is p-regular, and therefore (X, Y) G Co by Corollary 14.31 

3. So we may further assume that X and Y are both nilpotent. If 
X commutes with a non-zero semisimple element Z E p, then the same 
argument as in 1) works because the set of non-nilpotent elements is open. 
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4. Recall that an element of p is said to be p- distinguished if its cen- 
tralize! in p does not contain any non-zero semisimple element. In partic- 
ular, a p-distinguished element is nilpotent. So the number of X-orbits of 
p-distinguished elements is finite. 

Denote by tti : C(p) p the projection {Xi,X2) ^ Xi, O the set 
of non p-distinguished elements in p, and fii, . . . , the set of i^-orbits of 
p-distinguished elements in p. Thus p = O U VtiU ■ ■ ■ U fi^, and C(p) = 

Tll\0) U 'K^\^l) U ■ ■ • U Tl^\^lr). 

From the previous paragraph, we obtain that 7rf^((9) C Cq. Conse- 
quently, C(p) is the union of Cq with vrf ^(fijj, . . . , vrf ^(fijj. Now we check 
easily that for X G p, ti^^{K.X) = K.{X, p^) is an irreducible subset of C(p) 
of dimension dim 6 — dim 6"^ + dimp"^ = dim p. 

It follows that all irreducible components of C(p) other than Co, if they 
exist, are of dimension dim p. 

Suppose that C(p) is not irreducible. By the previous discussion, there 
exists a p-distinguished element X such that 7!-^^{K.X) is an irreducible 
component of dimension dim p. 

If X is even, then by Proposition l4.4^ X belongs to a K-sheet S containing 
non- zero semisimple elements. So dim 5 > dimK.X. Now Lemma (4.21 says 
that dim7rf^(5) > dimp. Contradiction. 

So we may assume that X is not even and the theorem follows from the 
following result. ■ 

Proposition 5.3 All p-distinguished elements of the symmetric (sOp+2, sOp x 
SO2), P > 2, are even. 

We shall first classify the nilpotent i^'-orbits in p. 



• Each element X of g can be represented by a matrix of the following 

type : 




The involutive automorphism 9 is defined by : 



9iX) 




It follows that : 
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The dimension of p is therefore 2p. 
Set : 

Hi = Ei^n-i+l ~ En-i-s^l^i , 1 < i < 2. 

The subspace a, spanned by Hi,H2, is a Cartan subspace of p. 

• Let Af be the set of G-nilpotent orbits in g, YDp+2 the set of all Young 
diagrams corresponding to the partitions of p + 2 and satisfying the two 
following properties : 

(Pi) Each row of even length occurs with even multiplicity. 

(P2) If all rows have even length, then Roman numerals are attached to the 
diagram, namely I or II. 

Recall (see for example "2]) that A/" is in one-to-one correspondence with 
YDp+2. 

• Let Ap be the set of nilpotent iC-orbits in p. To describe this set, we use 
the Kostant-Sekiguchi correspondence between Mp and the set A/q of nilpo- 
tent Go-orbits in Qq, where go is a real form of g and Go the corresponding 
connected real Lie group. Let us recall briefly this correspondence. 

We need to fix some notations : 

- go = so(p, 2) is the set of matrices Xq of the following type : 
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A Cartan decomposition of go is given by : 

00 = to © Po 

where the Cartan involution ^0 is defined by : ^o(-^o) = ~ *-^o- 



to = I ( ^2 J ' ^ SOp(M), ^2 e SO2 

Po = 1 ( t R n ) ' ^ ^ Mp,2 



- Go is the adjoint group of go. 
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The Lie algebra go is embedded in g, via the map (f) defined by: 



0(^O) 




where : 




Ip 
-ih 



) 



We check easily that : (j) o 9q = 6 o (jy. It follows that go can be identified 
with a real form of g. Hence, we have : 



• The next step is to describe Mo, using the method described in [21 
Chapter 9]. 

First of all, we define a signed Young diagram of signature {p, 2) to be a 
Young diagram in which each box is labeled with a + or a — sign in such a 
way that : 

- Signs alternate across rows. 

- Two diagrams are equivalent if and only if one can be obtained from 
the other by interchanging rows of equal length. 

- The number of boxes labeled + is p, the number of boxes labeled — is 



We define an orthogonal signed Young diagram of signature {p, 2) to be 
a signed Young diagram satisfying (Pi), (P2) and the three following prop- 
erties : 

(P3) Each row of even length has its leftmost box labeled +. 

(P4) If all rows have even length, then two Roman numerals, each I or II, 
are attached to the corresponding diagram (giving then four orbits). 

(P5) If at least one row has odd length and if each row of odd length has 
an even number of boxes labeled + or if each row of odd length has an 
even number of boxes labeled — , then one numeral I or II is attached 
to the corresponding diagram. 



(fio)c = 6, (po)c = P- 



2. 
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Let us denote by DY0p^2 the set of all orthogonal signed Young diagram. 
By a classical result (see [21 9.3.4]), we know that Aq is in one-to-one 
correspondence with DY0p_2- 

Remark 5.4 If Qq G A/q, then Q = {Qo)c £ A/". The Young diagram cor- 
responding to Q is obtained by removing the signs in the orthogonal signed 
Young diagram corresponding to Qq. 

Remark 5.5 It follows from the definition of elements of DY0p^2 that if 
^0 G A/q and fio = (rfi, . . . , (i^), then di < 5. 

• We may deduce now, from the previous statements, all the nilpotent 
orbits in so{p, 2). 

If p > 4, we have : 



+_ 

+ 



+ 



+ 



+ 





+ 




+ 




+ 




+ 




+ 




(5, !,...,!), 


+ 





(3,3,1,...,!), 
(I or II if p = 4) 



+ 



(3,1,1,...,!), 



+ 



+ 



+ 



(3,1,1 
I or II. 



,■■■,-^75 



(2,2,1,. ..,1). 
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If p = 3, the orbits are given by the partitions (5), (3, 1), (2, 2, 1) and the 
corresponding diagrams above. 

If p = 2, we have only the partitions (3, 1) and (2, 2). 

• Let ^0 G A/q. We may choose a sl2-triple {Hq, Xq,Yo) such that : 

Xq e fio 1 do{Ho) = —Hq , OoiXo) = —Yq , ^o(^o) = — -'^o- 

Triples with these properties are called Cayley triples. 
Set : 

Hs = i{Xo-Yo), Xs = ^{Xo + Yo + tHo) , Ys = ^{X^ + Yo - tHo) 

Then {Hs, Xs,Ys) is a normal sl2-triple (that is, Hs G t and Xs, Ys G p), 
called the Cayley transform of {Hq,Xq,Yq). 

Theorem 5.6 (Kostant-Sekiguchi correspondence [2J) There is a bi- 
jection from A/q onto A/'p which sends the orbit of the positive element of a 
Cayley triple to the orbit of the positive element of its Cayley transform. 

Via this correspondence, we classify the nilpotent X-orbits in p using the 
set DY0p,2- 

> + 2 " 
2 

of Q, where t is a Cartan subalgebra of the centralizer of a in 1^. We may 
complete the basis {Hi,H2) of a to a basis {Hi),l < i < r, of I), where 
{Hi, 3 < 2 < r), is a basis of t. Let {ei)i<i<r be the corresponding dual basis 
in [)*, R = R{q, [)) the corresponding root system and n a base of simple 
roots of R, defined as follows : 

j9 + 2 = 2r + l, R = {±ei ± Cj,! < i ^ j < r} U {ie^, 1 < i < r} 
U = {ci — Cj, l<i<j<r}U {cr} 
p + 2 = 2r, R = {±ei ±ej,l<i^ j <r} 

U = {Ci — Cj, l<i<j<r}U {Cr-l + Cr} 

In the same way, we choose a Cartan subalgebra f)o of Qq, such that 
f) = (f)o)c- In particular, ao = MHi © MH2 is a Cartan subspace of po such 
that a = (ao)c- 

• Given a nilpotent orbit Q in A/", there is a unique sl2-triple {h'^,e,f) 
such that : e G and for all a, G LI, ai{h'^) G {0, 1, 2}. 



Let r 



be the rank of q and f) = a © t a Cartan subalgebra 
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Recall that the sequence C{Q) — {ai{h'^), . . . , ar{h'^)) is called the char- 
acteristic of Q. 

Given a Young diagram, we can compute now the characteristic of the 
corresponding nilpotent orbit Q by using the following method : 

- First of all, we suppose that p + 2 = 2r + 1. Let Q = {di, . . . , dg) be 
a nilpotent G-orbit. Wc consider the sequence of integers (dj — 1, — 
3, . . . , —di + 1), 1 < i < s, and we rearrange it so that a comes first, 
followed by the other positive terms in non-increasing order, followed 
by the negative terms. Hence, the new sequence takes the form : 

(0, hi, /l2, • • • , hr, —hi, . . . , —hr) , hi > h2 > ■ ■ ■ > hr- 

Then, we have : 

C(0) = {hi - h2, h2- h^,..., hr-i - hr, hr). 

- Suppose now that p + 2 = 2r. 

If {di, . . . , dg) does not contain even terms, we use the same recipe as 
in the previous case and we obtain the sequence : 

{hi, /i2, • • • , hr, —hi, . . . , —hr) , hi > h2 > ■ ■ ■ > hr- 

Then, we have : 

C{fl) = {hi — h2, ■ . ■ , hr-l — hr, hr-l + hr) ■ 

If {di, . . . , ds) contains only even terms, we have the same sequence : 

{hi, /l2, . . . , hr, —hi, . . . , —hr) , hi > h2 > ■ ■ ■ > hr- 

Then, we set : a = 0, if r is a multiple of 4, and a = 2, otherwise. In 
this situation, there are two orbits fl^ and fl^^ . We have : 

C{n^) = {hi - h2, . - - , hr-2 - hr-l, a, 2 - a), 
C{Q.^^) — {hi — h2, - - - , hr-2 — hr-l, 2 — a, a). 

Clearly, a nilpotent element is even if all the terms of its characteristic 
are even. 
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Proof, (of Proposition I5.3|) Let X be a nilpotent element of p, Qk = K.X, 
Q = G.X and Qq = Gq.Xq, the corresponding real nilpotent orbit, via the 
Kostant-Sekiguchi correspondence. By |2l Remark 9.5.2], we have G.Xq = Q. 

Let {di, d2, . . . , ds) be the Young diagram of Q. By Remark 15.41 and the 
description of nilpotent orbits in so(p, 2), we only need to consider the two 
following cases : 

- {di,...,ds) 7^ (2,2,1,...,!). Hence, {di,...,ds) contains only odd 
terms. We deduce from the discussions above that C{Q) contains only 
even terms, and therefore, X is even. 

- (di, . . . , (is) = (2, 2, 1, . . . , 1). The two distinct corresponding real orbits 
are those of minimal dimension. Defining, from (cj), 1 < i < 2, a 
system of restricted roots for go, "we know by a classical result (see |21 
§4.3]) that these two orbits are respectively generated by Xq = Xe^-ej 
and —Xq. Consider now the element Hq = He^j^e2- Then, Hq is an 
element of p belonging to the centralizer of Xq. Applying the Cayley 
transform, we check easily that Hq E . So X is not p-distinguished. 

We have therefore proved that p-distinguished elements are even. ■ 

6 Explicit computations 

We shall describe in this section some of the explicit computations stated in 
the proofs of Lemmas 13.31 13.41 and Remarks 13.51 13.81 

• Case (i?3, tti). We have S = {02, ^s}, and 

Rs = «i + "2, ai + a2 + as, ai + 02 + 20:3, ai + 2a;2 + 203}. 

A simple computation shows that X^ai} and Xn are not ps-regular. 
For A G C*, set Xx = + XXu- Let F G pf \ 

Writing Y = Yi + Y_i with Yi G U5 and Y^i G u^, we conclude that 
Y = A + Z where A G a and Z G p^ = 0„gj^o 0„ with = (i?^ U -R^) \ 

{ai, -ai,en, -^n}- 

So we may assume that F G pg. It follows that 

= [Xx,Y] = [x„,,y_i] + A[x_,,,yi] + [x_,,,yi] + A[x,„,y_i]. 
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So 

Hence if F = Z^aeii? ^a^a, then [X^, F] = is equivalent to 

C7 [X_a J , X^] + Ac_/3 [Xgjj , X_^] = . 

whenever we have /?, 7 e i?^ fl i?° verifying /3 + 7 = ai + £n- The hst of such 
pairs is as follows: 

71) = (cKi + a2, Oil + q;2 + 2013) , (/32, /32) = («! + Q;2 + as, cti + q;2 + as)- 
Using the Chevalley basis from Gap4, we are reduced to the condition: 

c_/3j + Ac-yj = = c-y^ + Ac_^^ , 

These conditions can only be satisfied ifF = OorA = ±l. 

We have therefore checked that if X e a \ {0} is not p5-regular, then it 
belongs to one of the following lines : CX{ai}, CXn, CXi, CX_i. 

Using Gap4, we obtain that if X belongs to the first two lines, then 
dimg"^ = 7, thus its semisimple part is necessarily of type Ai x Ai. If X 
belongs to the other two lines, then dimg^ = 11, and its semisimple part 
must be of type B2. 

• Case (D5, ai). The elements of R\ are: 

CKi, CCi + 0:2, Q!i + 0:2 + 0:3, Q!i + 0:2 + 0:3 + 0:4, CCi + Q!2 + 0:3 + Qis, 

CKi + q;2 + 03 + "4 + «! + "2 + 2q;3 + q;4 + 0:5, 
cti + 2q!2 + 2q;3 + q;4 + 0:5. 

Using the same argument above, the pairs are 

(«! + q;2, ai + q;2 + 2^3 + 0:4 + q;5), 
(cKi + q;2 + Oiz, «! + q;2 + Q!3 + Q;4 + as); 
(cKi + q;2 + q;3 + a4, cki + 0:2 + as + as). 

Again, we may check using Gap4 that if X G a \ {0} is not p5-regular, then 
it belongs to one of the following hues : CXj^^j}, CXn, CXi, CX_i. 
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Using Gap4, we obtain that if X belongs to the first two fines, tfien 
dimg^ = 19, tfius its semisimple part is necessarily of type Ai x A^. If X 
belongs to tfie otfier two lines, tfien dimg"^ = 29, and its semisimple part 
must be of type D4. 

• Computations are similar for cases (so6,so4 x S02) and (so4,so2 x S02). 

• Case {An, 0:2) or (y4„, a„_i) witfi n > 4. Note tfiat tfie case tt, = 3 is just 
tfie symmetric pair (so6,so4 x S02). By symmetry, we only need to consider 
tfie case 02- 

Here £ consists of 11 and K = {a2, 03, . . . , a„_i}. Consider tfie element 
Xk- Its centralizer contains X^^ and Xq^+q.^ because n > 4. Since tfiese two 
root vectors are not proportional, tfie subpair (r, r+) associated to Xk can 
not be of type (som+i, som) (see jTTJ Proposition 3]). 

Via Gap4 computations, we see tfiat tfie semisimple part of is of 
type A2. 

• Case (1^5, 0:4) or (D^ja^). By symmetry, we may take a^. RecaU tfiat 
tfie two elements of S are K = {as, 04, 05} and 11. Consider tfie element 
Xk- Its centralizer contains X^^^ and Xoi+aj+as+as wfiicfi are not propor- 
tional. Consequently, tfie subpair (r, r+) associated to Xk can not be of type 

• Case {EQ,ai) or {EQ,aQ). Again we may take ai. Tfie set Rf fias 16 
elements. Tfie two elements of S are K = Il \ {02} and 11. Again, consider 
tfie element Xk- Its centralizer contains X^^^ and Xa-i^^a2+a3+a4- So tfie 
subpair (r, r+) associated to Xk can not be of type (sOm+i,sOm). 

Gap4 computations sfiow tfiat tfie semisimple part of g"^^' is of type A5. 

7 A final remark 

In view of Lemma f3.7t we tried to apply tfie same metfiod for tfie symmetric 
pair {Ey,Eq x C). Unfortunately, using tfie classification of p-distinguisfied 
orbits in p as described in jH], we found a non even p-distinguisfied element. 
Namely, tfie orbit corresponding to label 3 of Table 13 of [0]. 

Acknowledgements. Tfie autfiors would like to tfiank Patrice Tauvel and 
Abderrazak Bouaziz for many useful discussions. 
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